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‘THE (DANTZIG) SIMPLEX METHOD 
FOR LINEAR PROGRAMMING 


George Dantzig created a simplex algorithm to solve linear programs for planning and 
decision-making in large-scale enterprises. The algorithm’s success led to a vast array of 
specializations and generalizations that have dominated practical operations research for 
half a century. 


eorge Dantzig, in describing the 

“Origins of the Simplex Method,”! 

noted that it was the availability of 

early digital computers that sup- 
ported and invited the development of LP mod- 
els to solve real-world problems. He agreed that 
invention is sometimes the mother of necessity. 
Moreover, he commented that he initially re- 
jected the simplex method because it seemed in- 
tuitively more attractive to pursue the objective 
function downhill—as in the currently popular 
interior-point methods—rather than search 
along the constraint set’s edges. It is this latter 
approach that the simplex method uses, which 
should not be confused with the J.A. Nelder 
and R. Mead’s function-minimization method,” 
also associated with the word simplex. 


When Dantzig introduced his method in 1947, 
it was somewhat easier to sort out the details of 
the simplex method than to deal with the “where 
are we in the domain space” questions that are, in 
my opinion, the core of interior-point ap- 
proaches. Easier does not mean simpler, however. 


Chapter and verse 


The LP problem is, in one of the simplex 
method’s many forms, 


Minimize (with respect to X) € X (la) 
subject to Ax=b (1b) 
and x20 (1c) 


This is not always the problem we want to 
solve, though—we might want to have the ma- 
trix A and vector b partitioned row-wise so 


that 
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Minimize (with respect tox) CX (2a) 


subject to Al x<hbl (2b) 
A2x=h2 (2c) 

A3 x> b3 (2d) 

x20 (2e) 


However, the use of slack and surplus vari- 
ables (the X variables we use to augment our 
problem) can transform the problems from one 
to the other. In fact, we must add slacks to the 
left-hand side of Equation 2b and subtract them 
from Equation 2d to arrive at equalities. More- 
over, the special non-negativity conditions in 
Equations 1c and 2e could be subsumed into 
the matrix equation or inequation structure, but 
the LP tradition calls for listing them sepa- 
rately. The number of rows in A apart from the 
non-negativity constraints we call 7, and the 
number of variables (including the slacks and 
surpluses) we call n. Typically, m < n. 

The simplex method assumes we have an ini- 
tial basic feasible solution X;,;,, that is, a solution 
that is feasible and that has just of the x’s non- 
zero. The mX values multiply m columns of A, 
which we call the basis. “Basic” definitely has a 
technical rather than a general meaning here. 
We assume this set of basis vectors (columns of 
A) is linearly independent and that it has full 
rank. Therefore, the core of the simplex method 
is to exchange one of the columns of A that is in 
the basis for one that is not. This corresponds 
to increasing one of the nonbasic variables while 
keeping the constraints satisfied, a process that 
reduces some of the x’s. We choose to increase 
the x that gives us the most decrease in the ob- 
jective function, and we continue to increase it 
until one of the current basic variables goes to 
zero, which means we now have a new basis. 
Moreover, the new solution turns out to be a 
new vertex of the simplex that the constraints 
define, and it is also a neighboring vertex to that 
described by our starting basic feasible solution. 
We have exchanged one vertex of the simplex 
for one of its neighbors and done so in a way 
that moves the objective function toward the 
minimum. 

This exchange, or pivoting, seems to be a topic 
especially devised to torture business school un- 
dergraduates. Nevertheless, it is theoretically 
and practically a relatively simple process, un- 
less, of course, you have a computer without 
enough memory to hold everything all at once. 
Additionally, there are some nasty details we 
have to sort out: 


e How do we get an initial feasible solution? 
Or, more importantly, how do we tell if 
there is no feasible solution? 

e What happens if the constraints don’t stop 
the objective from being reduced? That is, 
what if the solution is unbounded? 

e What happens if several vertices give equal 
objective values? 


‘These questions, and the memory-management 
issues, occupied many researchers and generated 
many journal and book pages over the years. For 
example, more than half the pages in S. Gass’s 
classic text are devoted to such details;’ the ap- 
plications come so late in the book that students 
of slow lecturers must have wondered what all 
the fuss was about. 

Researchers handled the questions mentioned 
earlier in a variety of ways. Clever management 
and programming tactics overcame the memory 
issue. In early years, storage was so slow that pro- 
grammers spent considerable effort to arrange 
computations that would complete in time to 
read and write data as disks and drums presented 
the appropriate tracks and sectors. Such compli- 
cations help to obscure the central ideas of the 
algorithms in a muddle of detail, mostly irrele- 
vant now. 

The initial feasible solution issue was addressed, 
perhaps surprisingly, by adding 7 more variables, 
called artificial variables, which form the initial 
basis, then minimizing an objective function that 
drives them all out of the basis.* If we cannot drive 
them out, we have an infeasible problem. 

Unbounded solutions turn out to be rather 
simple to detect, but the “equal objective func- 
tion” or degeneracy issue worried people a great 
deal, because they implied the possibility that the 
algorithm might not terminate. In practice, per- 
turbation or rule-based methods can avoid the 


cycling. 


Nice extras 


Modern textbooks spend much less ink on the 
details.’ Similarly, recent discussions® of the 
computational techniques have abandoned the 
tableaux and the jargon of earlier work** re- 
garding the matrix notations of numerical linear 
algebra. What remains of great interest, both 
mathematically and computationally, is the du- 
ality feature of LP problems. That is, an LP 
written as 


(Primal) minimize € X such that A x> b, x>0 


COMPUTING IN SCIENCE & ENGINEERING 


has a dual equivalent 
(Dual) maximize b’y such that A’y<c y2 0. 


Moreover, the optima, if they exist, have the 
same objective value. Duality also leads to use- 
ful interpretations of solutions in terms of prices 
or values when the LP problems under consid- 
eration have economic or related contexts. 


Practical importance 


The simplex method’s importance really lies 
in the value of the LP applications, even when 
the LP model is only a crude approximation to 
the real world. In the 1940s, many organizations 
were very hungry for solutions to LP problems, 
even if they did not realize what LP was. Oil and 
chemical companies led the way, especially for 
optimizing product mix from multiple sources 
or multiple sites. Transportation companies and 
the military recognized quite early that trans- 
portation and logistics problems could be for- 
mulated as LPs. Large-scale agricultural eco- 
nomic problems were also early applications. 
Examples also became realized in production 
planning, staff and resource scheduling, and net- 
work or traffic flows. LP can even be applied to 
maintaining the confidentiality of government 
statistics.’ 

For example, the Diet Problem has as objec- 
tive function elements (©) the costs per unit of 
food ingredients. The right-hand side values (b) 
are the required minimum amounts of each of a 
list of m nutrients, and the constraint coefficients 
A give the amount of each nutrient in a unit of 
each ingredient. Thus we want to have A x 2 bto 
satisfy the nutrient requirements (we could also 
put in upper limits of nutrients like vitamin A 
that can be toxic), but we also want the cheapest 
blend. Of course, the resulting solution might 
not be very tasty. It is simply nutritious, and by 
construction will be the cheapest such recipe. 
Perhaps this is the origin of institutional cafete- 
ria food. 


Ithough the mathematics and compu- 
tational algorithms might be fascinat- 
ing, my view is that the value of the 
applications is the first reason for the 
importance of the simplex method. Promotion 
to the “top 10” category still needs something 


else, and this is the particular efficiency of the 
simplex method in finding the best vertex of the 
simplex. Most constraint matrices in practical 
LP problems are quite sparse, and the number 
of iterations where we move from one vertex to 
another is generally very small relative to the 
number of variables x. Even better, we can ex- 
plore the sensitivity of the optimal solution to 
small changes in the constraint and objective 
very easily. This is not always the case for some 
other methods. 

Did Dantzig realize this efficiency as he first 
coded the simplex method? My guess is that he 
did not, although it is clear from his reminis- 
cences! that such an understanding was not long 
in coming. In looking back over half a century, 
I find it remarkable that so many workers could 
sort through the jungle of awkward, and to some 
extent unnecessary, details to see the underlying 
tool’s value. $è 
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